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ABSTRACT 

Students continue to demand and enroll in online courses, but are not always satisfied with their 
experiences. The purpose of this study was to determine if students’ responses to evaluations for online 
courses could be used to identify faculty actions that could lead to improved evaluation scores in teaching 
effectiveness and overall course value. Controversy continues to exist over the validity of student 
evaluations to measure faculty effectiveness and overall course quality. Faculty do not always utilize the 
collected data for the improvement of teaching. Results indicate that stimulation of learning had the most 
effect on perceptions of teaching effectiveness and useful and relevant assignments had the highest 
correlation to overall course value. 
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I. INTRODUCTION 

Student demands for distance learning courses, specifically those delivered online continue to grow. A 
2008 study conducted of distance learning in community colleges by the Instructional Technology 
Council (ITC) found that: 

The rate of growth for distance education (an 11 percent increase for 2006-2007) far outpaced the 
growth rate for traditional enrollments. Seventy percent of the respondents reported that student 
demand exceeds current class offerings. [1, p. 9] 

The National Center for Education Statistics supported the findings of the ITC, reporting that there were 
12.2 million enrollments in distance education courses in the 2006-2007 academic year at 2- and 4-year 
institutions, with 77% of those in online courses [2]. 

College and universities are utilizing distance learning technologies to increase access to their courses and 
degree offerings, to better serve the needs of their students, as well as to increase their revenues in a 
downed economy and decreasing state funding. The strategic move to increase the number of distance 
courses offered leads to the expectation that more college and university faculty will be expected to teach 
online. 

The quality of online courses continues to be debated extensively in the literature, with little consensus on 
how to measure it, as well as how to assess teaching effectiveness of faculty teaching these courses. One 
traditional instrument of assessment that has historically been utilized to assess students' perceptions of 
teaching effectiveness and overall course value is the student course evaluation. Course evaluations 
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appear to be the most accepted method of assessment for teaching competencies of faculty at most 
colleges and universities [3]. The results of these assessments are oftentimes utilized for promotion and 
tenure decisions, merit pay increases, as well as future employment decisions [3, 4]. 

II. RELATED LITERATURE 

There are opposing views of the usefulness of student course evaluations to assess teaching effectiveness. 
Opponents of the usefulness of student evaluations cite that college students cannot be expected to 
understand nor appreciate the significance of what they are undergoing in a course until later in their 
careers as their perspectives mature [5] and students' lack knowledge of the aspects of teaching [6]. 
Proponents of course evaluations support that students are a good source to provide feedback on teaching 
[3], as they are the receivers of the course experience. Knowing the importance of these assessments to 
the careers of college and university faculty, this study seeks to determine if the data collected through 
student evaluations of online courses can be used to identify actions that can be taken by faculty to 
improve students' perceptions of teaching effectiveness and overall course value. 

The literature is rich in studies that address faculty behaviors that lead to perceived student satisfaction in 
both the traditional and online environments. Much of this research is grounded in the work of Chickering 
and Gamson's [7] Seven Principles of Effective Teaching. Student satisfaction within the online classroom 
is directly affected by quantity and quality of interactions between the assigned faculty member and the 
enrolled online students [8, 9, 10, 11], A study conducted by Arbaugh [12] of 25 online sections in a 
graduate program, found that "immediacy behaviors" of professors correlated with student satisfaction in 
the online environment, more so than student satisfaction with an instructor’s mastery of technology [13]. 

Shea et al. [10] discovered that the expression of clear expectations and timely feedback also directly 
impacted student satisfaction. As noted above, increased clarity of expression and timeliness of feedback 
resulted in increased online student satisfaction. Additional factors, which the researchers identified as 
positively affecting satisfaction among online students, were low levels of technical difficulties and high 
quality feedback on assignments. 

A study conducted by Young analyzed students' views of effective online teaching. Seven core items 
were identified: "adapting to student needs, using meaningful examples, motivating students to do their 
best, facilitating the course effectively, delivering a valuable course, communicating effectively, and 
showing concern for student learning" [14, p. 65]. 

Northrup [15] assessed the needs of online students and found that those who could regulate their own 
learning and received timely responses from instructors were more satisfied with the online course 
experience. Tricker, Rangecroft, Long, and Gilroy [16] evaluated a variety of distance education courses 
and found that students were attracted to online courses because of their flexibility. Though flexibility 
was the initial attraction, students still expected a high quality course, with assignments and course 
materials that were professionally meaningful. In addition, they wanted high-quality feedback and good 
communication from their instructors. 

Young's [14] study also identified students' perceptions of ineffective and effective online faculty. 
Students' perceptions of ineffective online faculty were those that were not involved in class discussions, 
who did not communicate often and effectively with students, and who did not provide students prompt 
feedback on their work. 

A study conducted by Spangle, Hodne, and Schierling [17] examined more than 1,200 student evaluations 
of online courses along with surveys of the instructors and found that effective online faculty: 

• Have good written communication skills; 

• Have carefully designed course activities that promote discussion; 

• Provide timely feedback; 

• Are flexible with students, adapting to their various needs; 

• Demand high-quality work; and 
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• Create an atmosphere that encourages students to collaborate and interact with their classmates, 
their instructor, and course materials. 

III. THE STUDY 

The purpose of this study was to determine if students’ responses to evaluations for online courses could 
be used to identify faculty actions that could lead to improved scores in teaching effectiveness and overall 
course value. Controversy continues to exist over the validity of student evaluations to measure faculty 
effectiveness and overall course quality [4, 5]. Typical student evaluations are based on Likert scales and 
generally result in an overall average based on multiple categories of information. Student course 
evaluation data are not often used for overall quality improvement of teaching, and oftentimes are 
provided to faculty who perhaps do not utilize the information as possible predictors of how to improve 
their teaching effectiveness. Generally, course evaluations are "seen as an end in themselves and their 
numbers used, without any other analysis of interpretation, to make important personnel decisions," [3, p. 
56] such as for promotion and tenure, as well as merit increases for faculty [4]. Two research questions 
guided this study: 

1. What faculty behaviors in online courses lead to higher perceptions of student satisfaction of 
overall instructor effectiveness? 

2. How do faculty actions in online courses affect student perceptions of the overall value of the 
online course? 

A. Research Design 

The data analyzed for this study were all student course evaluations for fully online courses from one 
large, research university located in the southwestern region of the United States. The university has a 
current student enrollment of over 30,000. In 2010, the university had 3,890 students enrolled in online 
courses (more than 50% of the course being delivered online), with 356 courses offered. The data set for 
this study was online course evaluations collected between spring 2007 to fall 2009. There were a total of 
2,826 records. The evaluations were for both undergraduate and graduate-level courses. The online 
course evaluation instrument was a university-designed assessment that contained 23-questions (see Table 
1) and is different than the evaluation used for on-campus courses. Ten of the questions addressed 
instructor actions leading to perceptions of teaching effectiveness, ten addressed overall course value, and 
the remaining questions related to college and university technical support, student advising, and 
registration procedures. 


_ Question _ 

Overall the instructor(s) was (were) effective. 

The instructor(s) was (were) available for 
consultation in a timely manner. 

The instructor(s) stimulated student learning. 

The instructor(s) treated me fairly. 

The instructor(s) treated me with respect. 

The instructor(s) welcomed and encouraged questions and comments. 
The instructor(s) presented the information clearly. 

The instructor(s) emphasized the major points and concepts. 

The instructor(s) went beyond presenting the information in the text. 
The instructor(s) demonstrated knowledge of the subject. 

Overall this course was a valuable learning experience. 

The assignments were relevant and useful. 


_ Measures _ 

Teaching Effectiveness 
Teaching Effectiveness 

Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Teaching Effectiveness 
Course Value 
Course Value 
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Course materials were relevant and useful. 

Expectations were clearly stated either verbally or in the syllabus. 

The testing and evaluation procedures were fair. 

The workload was appropriate for the hours of credit. 

The textbook or other purchased materials were relevant and useful. 

The technology for delivering the instruction was appropriate for the 
course. 

Information was available to help me solve technical problems before 
they caused delays in my course 

Documentation for accessing electronically-delivered class lectures 
and/or material was effective. 

Registration procedures for the course ran smoothly. 

Advising was available and adequate. 

Course materials were readily accessible. _ 


Course Value 
Course Value 
Course Value 
Course Value 
Course Value 
Course Value 

Course Value 

Course Value 

Institution Procedures 
Institution Procedures 
Institution Procedures 


Table 1. Online Course Evaluation Questions 

To determine teaching effectiveness and course value, questions on teaching effectiveness and overall 
course value were analyzed to address the two research questions. Cronbach's alphas for the nine teaching 
effectiveness and nine overall course value items were .96 and .95, respectively. Descriptive statistics, 
Pearson correlation coefficients, and multiple linear regressions were used to analyze faculty actions that 
affected students' perceptions of teaching effectiveness and overall course value based on the results of 
the online course evaluations. 


1. Data Collection and Tools 

Descriptive statistical analysis, conducted using PASW 18.0, were used to provide a breakdown of the 
overall course evaluation means for the two questions driving this study: 1) Overall the instructor was 
effective, and 2) Overall the course was a valuable learning experience (see Table 2). These means were 
evaluated on a 5-point Likert scale with 1 = strongly disagree and 5 = strongly agree. 


Ouestion 

N 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Instructor Effectiveness 

2,826 

79 

160 

278 

1,044 

1,265 



(2.8%) 

(5.7%) 

(9.8%) 

(36.9%) 

(44.8%) 

Overall course was valuable 

2,826 

120 

159 

267 

979 

1,302 



(4.2%) 

(5.6%) 

(9.4%) 

(34.6%) 

(46.1%) 


Table 2. Overall Course Evaluations for Instructor Effectiveness and Course Value 
The student course evaluations indicated high satisfaction overall with teaching effectiveness (81.7%) and 
the online course being valuable (80.7%), with students agreeing or strongly agreeing. To further dissect 
the evaluation data to determine what factors could be used to identify higher student perceptions of 
teaching effectiveness and overall course value, correlations and multiple linear regressions were 
conducted on multiple independent variables. 

2. Analysis and Findings 

a. Teaching Effectiveness 

Analysis using the Pearson's correlation coefficient indicated that there were positive correlations 
between teaching effectiveness and nine independent predictor variables. The highest correlations were 
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with stimulation of student learning (r = .850), information presented clearly (r = .806), major points and 
concepts were emphasized (r = .786), information was presented beyond the textbook (r = .769), and 
assignments were relevant and useful (r = .748). Initial correlation analysis indicated that there was less 
than a 1% chance (p < .01) that the relationships occurred by chance (please see Table 3 for further 
breakdowns of correlations). 


Correlation 


Independent Variable 

Coefficient 

Sig. (1-tailed) 

N 

Stimulated Student Learning 

.850 

.000 

2,826 

Presented Information Clearly 

.806 

.000 

2,826 

Emphasized Major Points & Concepts 

.786 

.000 

2,826 

Went Beyond Presenting Info in Text 

.769 

.000 

2,826 

Demonstrated knowledge of Subject 

.735 

.000 

2,826 

Available in a Timely Manner 

.723 

.000 

2,826 

Welcomed & Encouraged Questions 

.722 

.000 

2,826 

Treated Students Fairly 

.705 

.000 

2,826 

Treated Students Respectfully 

.691 

.000 

2,826 


Table 3. Correlation Analysis of Faculty Actions and Dependent Variable “Teaching Effectiveness” 

Based upon the close relationships identified between multiple independent variables and the dependent 
variable, further study using multiple regression analysis was necessary to identify the amount of variance 
in the dependent variable, teaching effectiveness, which could be attributed to each independent variable. 
Using the Forward method for multiple regression analysis, independent variables were entered into the 
equation and assessed for significance by the program as identified in Tables 4 and 5. The predictors were 
entered into the equation at a .05 level of significance, and removed from the equation if the significance 
fell below .10. Missing cases were pairwise deleted. 

The multiple R indicated a high positive correlation between eight of the predictor variables and the 
dependent variable (R = .898). The R Square value for Model 5 (r 2 = .806) revealed that approximately 
80% of the variance in the dependent variable, teaching effectiveness, could be explained by eight 
independent variables. 


Model 

R 

R Square 

Adjusted 

R Square 

Std. Error 

of the Estimate 

8 

.898 

.806 

.805 

.442 


Table 4. Multiple Regression Analysis Summary - Predictors: (constant = teaching effectiveness), stimulated learning, 
presented information clearly, emphasized major points, demonstrated knowledge, was available, treated students fairly, 

welcomed and encouraged questions, went beyond textbook 

The degree of influence of each predictor variable on teaching effectiveness was identified by the Beta 
value. Instructor stimulation of learning accounted for .38 of the defined variance (B = .378), and 
information presented clearly (B = .212) accounted for .23 of the variance. The instructor being available 
(B = .103) explained .11 of the identified variance, while instructor treated students fairly (B = .109) 
accounted for .09 of the variance of student perceptions of teaching effectiveness. Table 5 provides a 
breakdown of the variances of perceptions of teaching effectiveness for all eight predictor variables. The 
independent variable, treated students with respect, was excluded by the PASW statistical program during 
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the Forward method analysis as being non-significant, accounting for less than 1 percent of the variance 
associated with perceptions of teaching effectiveness. 


Unstandardized Standardized 

Coefficients Coefficients 


Predictor Variable 

B 

Std. Error 

Beta 

t 

Sig. 

Stimulated Learning 

.378 

.016 

.383 

22.973 

.000 

Presented Info Clearly 

.212 

.015 

.229 

14.017 

.000 

Was Available 

.103 

.013 

.107 

7.988 

.000 

Treated Students Fairly 

.109 

.017 

.085 

6.364 

.000 

Demonstrated knowledge 

TOO 

.016 

.088 

6.155 

.000 

Emphasized Major Points 

.057 

.018 

.055 

3.155 

.002 

Went Beyond Textbook 

.032 

.015 

.035 

2.168 

.030 

Welcomed Questions 

.034 

016 

.031 

2.150 

.032 


Table 5. Degrees of Influence of Predictor Variables of Faculty Actions on Dependent Variable “Teaching Effectiveness” 

b. Overall Course Value 

Analysis using the Pearson's correlation coefficient indicated that there were high positive correlations 
between perceived overall course value and assignment usefulness and relevancy (r = .808) and course 
materials' usefulness and relevancy (r = .787). Moderate correlations existed between overall course 
value and fair testing and evaluation procedures (r = .665), and usefulness and relevancy of purchased 
materials (r = .652). Initial correlation analysis indicated that there was less than a 1% chance (p < .01) 
that the relationships occurred by chance (please see Table 6 for further breakdowns of correlations). 


Independent Variable 

Correlation 

Coefficient 

Sig. (1-tailed) 

n 

Assignments Useful/Relevant 

.808 

.000 

2,826 

Course Materials Useful/Relevant 

.787 

.000 

2,826 

Testing/Evaluation Fair 

.665 

.000 

2,826 

Purchased Materials Useful/Relevant 

.652 

.000 

2,826 

Documentation for Course Navigation 

.616 

.000 

2,826 

was Effective 

Information was Available for 

.609 

.000 

2,826 

Technical Issues 

Expectations Clearly Stated 

.605 

.000 

2,826 

Technology Used was Appropriate 

.597 

.000 

2,826 

Workload was Appropriate 

.596 

.000 

2.826 


Table 6. Correlation Analysis of Course Procedures/Content and Dependent Variable “Overall Course Value” 
Based upon the close relationships identified between multiple independent variables and the dependent 
variable, further analysis using multiple linear regression was necessary to identify the amount of 
variance in the dependent variable, overall course value, which could be attributed to each independent 
variable. Using the Forward method for multiple regression analysis, the independent variables were 
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entered into the equation and assessed for significance by the program. The predictor variables were 
entered into the equation at a .05 level of significance, and removed from the equation if the significance 
fell below .10. Missing cases were pairwise deleted. The independent variables workload was appropriate 
for hours of credit and documentation for navigating the online course was effective, were excluded by 
the PASW statistical program during the Forward method analysis as being non-significant, accounting 
for less than 1 percent of the variance associated with perceptions of overall course value. 

The multiple R indicated a high positive correlation between seven of the predictor variables and the 
dependent variable (R = .848). The R Square value for Model 7 (r 2 = .719) revealed that approximately 
72% of the variance in the dependent variable, overall course value, could be explained by the seven 
independent variables (see Table 7). 


Model 

R 

R Square 

Adjusted 

R Square 

Std. Error 

of the Estimate 

7 

.848 

.719 

.718 

.568 


Table 7. Multiple Regression Analysis Summary - Predictors: (constant = overall course value) assignments were 
relevant/useful, course materials were relevant/useful, testing and evaluation were fair, information was available to solve 
technical issues, expectations were clear, technology used was appropriate, and purchased materials were relevant/useful 

The degree of influence of each predictor variable was identified by the Beta value. Assignments being 
relevant and useful accounted for .43 of the defined variance in the perception of course value (B = .434). 
The relevance and usefulness of course materials (B = .307) accounted for .38 of the variance. These two 
variables had the greatest influence on students' perception of overall course value. Table 8 provides a 
breakdown of the variances of perceptions of students of overall course value based on the independent 
variables. The independent variables, usefulness and relevancy of documentation for course navigation 
and workload appropriate for the hours of credit, were excluded by the PASW statistical program during 
the Forward method analysis as being non-significant, accounting for less than 1 percent of the variance 
associated with perceptions of overall course value. 


Predictor Variable 

Unstandardized 

Coefficients 

B 

Std. Error 

Standardized 

Coefficients 

Beta 

t 

Sig. 

Assignments Useful/Relevant 

.434 

.020 

.406 

21.781 

.000 

Course Materials Relevant/ 

.307 

.023 

.273 

13.489 

.000 

Useful 

Testing/Eval Procedures Fair 

.093 

.018 

.083 

5.211 

.000 

Information was Available to 

.070 

.017 

.066 

4.223 

.000 

Solve Technical Issues 

Expectations were Clear 

.065 

.016 

.059 

4.041 

.000 

Used Appropriate Technology 

.041 

.018 

.035 

2.307 

.000 

Purchased Materials Relevant/ 

.039 

.017 

.036 

2.284 

.009 


Useful 


Table 8. Degrees of Influence of Predictor Variables of Course Structure on Dependent Variable “Overall Course Value” 


Journal of Asynchronous Learning Networks, Volume 16: Issue 1 


55 



Reading Between the Lines of Online Course Evaluations: Predictable Actions that Improve Student 
Perceptions of Teaching Effectiveness and Course Value 


IV. DISCUSSION 

The purpose of this study was to determine if the data collected in student evaluations of online courses 
could be used to identify faculty actions that impacted students' perceptions of teaching effectiveness and 
overall course value. The results of the study indicate that data collected through student course 
evaluations for online courses can possibly provide useful information for faculty and higher education 
administrators to identify actions that can be taken to improve overall student satisfaction. As the 
literature extensively discusses, colleges and universities rely predominantly on student course 
evaluations to assess teaching competency. Though there is some support for students' abilities to gauge 
teaching effectiveness, there is equivalent support that students do not have a sufficient understanding of 
teaching to do so. In addition, there is debate that course evaluations are neither reliable nor valid and 
truly do not measure student learning, which is a better predictor of teaching effectiveness. Regardless of 
what one perceives of these assessment instruments, colleges and universities utilize them in their faculty 
tenure and promotion decisions, as well as merit pay increases. In addition, faculty seeking new positions 
are often asked to reproduce their student evaluations in their application packages to support their 
teaching effectiveness. Due to this extensive reliance on these assessments to measure teaching 
competence and future merit and promotion decisions, it is important for faculty to consider utilizing the 
information collected to improve their online courses to maximize their student evaluation scores. 

The results of the study identified specific faculty actions within online courses that appear to lead to 
higher levels of student perceptions of teaching effectiveness. The action that had the greatest impact was 
that faculty stimulated learning. Students in online courses want high quality and rigorous courses that are 
well developed and organized, and that provide them with engaging learning experiences. Students expect 
their online instructors to develop and deliver challenging and worthwhile courses that offer alternatives 
to the traditional classroom, but not at the risk of losing high-quality learning experiences. Students also 
appreciate faculty that make a strong effort to facilitate a thoughtful course that is organized and carefully 
structured. Simply stated, students want online faculty that are engaged in the course. This supports 
the findings of the prior research on student perceptions of effective online teaching [10, 14, 16, 17], 

The variable that had the second greatest impact was faculty presented information clearly. This entails 
their written communications, as well as the organization and development of their online courses. If 
students are not able to easily navigate their courses, they may feel lost and overwhelmed. It is important 
that all communications from faculty are well articulated and organized. The need for clear expectations 
and timely feedback impact student satisfaction [10]. Also important to the perceptions of teaching 
effectiveness are faculty availability, and how they support and treat students. It is important for faculty 
to understand that the online environment is daunting and isolated for many students. If students do not 
receive supportive and timely communications from faculty when they are having problems within the 
course, they can become dissatisfied, which could result in them withdrawing from their courses. 

The perceptions of overall online course value can be improved by ensuring that assignments and course 
materials chosen are relevant and useful. This supports findings by Young that identified that students 
found faculty effective when they utilized "meaningful examples" and delivered "a valuable course" [14, 
p. 65], Faculty must select course materials and design and develop their assignments to ensure that they 
contribute to the mastery of the learning outcomes of the course. Students want to know that what they 
are being required to do will increase their skills and knowledge. Some faculty perceive that additional 
readings and assignments are necessary to ensure online courses are rigorous. The results of this study 
and prior research indicate that the careful selection of relevant and useful assignments lead to higher 
levels of student satisfaction in online courses. 

The results of this study have identified factors that can be addressed in online courses to help improve 
student perceptions of teaching effectiveness and overall course value. The limitations to this study are 
that the course evaluation instrument utilized for analysis is specific to one large, research university. 
There is not a generalized course evaluation instrument utilized by all colleges and universities, so readers 
will need to assess the generalizability of these findings for their purposes. Also, the results of this study 
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are not presented to determine a cause-and-effect relationship through its findings. Further research 
should utilize qualitative methodologies to further evaluate students' needs in online courses and what 
they value as important to their experiences. 
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